An n-Sigmoid Activation Function to Improve the Squeeze-and-Excitation for 2D and 3D Deep Networks

The Squeeze-and-Excitation (SE) structure has been designed to enhance the neural network performance by allowing it execute positive channel-wise feature recalibration and suppress less useful features. SE structures are generally adopted in a plethora of tasks directly existing models have shown actual enhancements. However, various sigmoid functions used artificial networks intrinsically restricted vanishing gradients. purpose this paper is further improve introducing new block with custom activation function resulting from integration piecewise shifted function. proposed aims learning generalization capacity 2D 3D for classification segmentation, reducing gradient problem. Comparisons were made between original design, addition block, n-sigmoid block. To evaluate method, commonly datasets, CIFAR-10 Carvana data Sandstone Dataset data, considered. Experiments conducted using showed that results improvements training accuracy score UNet (up 0.25% 99.67%), ResNet 0.9% 95.1%), DenseNet 1.1% 98.87%) cases, 0.2% 99.67%) cases. not only reduces problem but also develops valuable features combining spatial information.